On Criteria for Formal Theory Building: Applying Logic and Automated Reasoning Tools to the Social Sciences

This paper provides practical operationalizations of criteria for evaluating scientific theories, such as the consistency and falsifiability of theories and the soundness of inferences, that take into account definitions. The precise formulation of these criteria is tailored to the use of automated theorem provers and automated model generators—generic tools from the field of automated reasoning. The use of these criteria is illustrated by applying them to a first order logic representation of a classic organization theory, Thompson’s Organizations in Action. Introduction Philosophy of science’s classical conception of scientific theories is based on the axiomatization of theories in (first order) logic. In such an axiomatization, the theory’s predictions can be derived as theorems by the inference rules of the logic. In practice, only very few theories from the empirical sciences have been formalized in first order logic. One of the reasons is that the calculations involved in formalizing scientific theories quickly defy manual processing. The availability of automated reasoning tools allows us to transcend these limitations. In the social sciences, this has led to renewed interest in the axiomatization of scientific theories (Péli et al. 1994; Péli & Masuch 1997; Péli 1997; Bruggeman 1997; Hannan 1998; Kamps & Pólos 1999). These authors present first order logic versions of heretofore non-formal scientific theories. The social sciences are renowned for the richness of their vocabulary (one of the most noticeable differences with theories in other sciences). Social science theories are usually stated using many related concepts that have subtle differences in meaning. As a result, a formal rendition of a social science theory will use a large vocabulary. We recently started to experiment with the use of definitions as a means to combine a rich vocabulary with a small number of primitive terms. Now definitions are unlike theorems and unlike axioms. Unlike theorems, definitions are not things we prove. We just declare them by fiat. But unlike axioms, we do not expect definitions to add substantive information. A definition is expected to add to our convenience, not to our knowledge. (Enderton 1972, p.154) Copyright c 1999, American Association for Artificial Intelligence (www.aaai.org). All rights reserved. If dependencies between different concepts are made explicit, we may be able to define some concepts in terms of other concepts, or in terms of a smaller number of primitive concepts. If the theory contains definitions, the defined concepts can be eliminated from the theory by expanding the definitions. Eliminating the defined concepts does not affect the theory, in the sense that the models and theorems of the theory remain the same. This paper provides practical operationalizations of criteria for evaluating scientific theories, such as the consistency and falsifiability of theories and the soundness of inferences. In earlier discussion of the criteria for evaluating theories, we did not distinguish between different types of premises (Kamps 1998). In this paper, we will provide practical operationalizations of these criteria that take definitions into account, and illustrate their use on a formal fragment of organization theory. Logical formalization Most social science theories are stated in ordinary language (except, of course, for mathematical theories in economics). The main obstacle for the formalization of such a discursive theory is their rational reconstruction: interpreting the text, distinguishing important claims and argumentation from other parts of the text, and reconstructing the argumentation. This reconstruction is seldom a straightforward process, although there are some useful guidelines (Fisher 1988). When the theoretical statements are singled-out, they can be formulated in first order logic. The main benefit of the formalization of theories in logic is that it provides clarity by providing an unambiguous exposition of the theory (Suppes 1968). Moreover, the fields of logic and philosophy of science have provided a number of criteria for evaluating formal theories, such as the consistency and falsifiability of theories and the soundness of inferences. Our aim is to develop support for the axiomatization of theories in first order logic by giving specific operationalizations of these criteria. These specific formulations are chosen such that the criteria can be established in practice with relative ease, i.e., such that existing automated reasoning tools can be used for this purpose.1 Of course, we hope that this will be regarded as an original contribution, but the claim to originality is a difficult one to establish. The novelty is in the combination of ideas from various fields and our debts to the fields of logic and philosophy of science fan out much further than specific citations indicate. Criteria for Evaluating Theories We will use the following notation. Let denote the set of premises of a theory. A formula is a theorem of this theory if and only if it is a logical consequence, i.e., if and only if . The theory itself is the set of all theorems, in symbols, . Consistency The first and foremost criterion is consistency: we can tell whether a theory in logic is contradictionfree. If a theory is inconsistent, it cannot correspond to its intended domain of application. Therefore, empirical testing should focus on identifying those premises that do not hold in its domain. The formal theory can suggest which assumptions are problematic by identifying (minimal) inconsistent subsets of the premises. The theory is consistent if we can find a model such that the premises are satisfied: . A theory is inconsistent if we can derive a contradiction, , from the premises: . Soundness Another criterion is soundness of arguments: we can tell whether a claim undeniably follows from the given premises. If the derivation of a claim is unsound, empirical testing of the premises does not provide conclusive support for the claim. Conversely, empirically refuting the claim may have no further consequences for the theory. Many of our basic propositions are inaccessible for direct empirical testing. Such propositions can be indirectly tested by their testable implications (Hempel 1966). In case of unsound argumentation, examining the counterexamples provides useful guidance for revision of the theory. A theorem is sound if it can be derived from the premises . A theorem is unsound (i.e., is no theorem) if we can construct a counterexample, that is, a model in which the premises hold, and the theorem is false: and . Falsifiability Falsifiability of a theorem means that it is possible to refute the theorem. Self-contained or tautological statements are unfalsifiable—their truth does not depend on the empirical assumptions of the theory. Falsifiability is an essential property of scientific theories (Popper 1959). If no state of affairs can falsify a theory, empirical testing can only reassert its trivial validity. A theory is falsifiable if it contains at least one falsifiable theorem. An initial operationalization of falsifiability is: a theorem is unfalsifiable if it can be derived from an empty set of premises: and falsifiable if we can construct a model (of the language) in which the theorem is false: . Note that we cannot require this model to be a model of the theory. A theorem is necessarily true in all models of the theory (otherwise it would not be a theorem). We should therefore ignore the axioms of the theory, and consider arbitrary models of the language. This initial formulation works for some unfalsifiable statements like tautologies, but may fail in the context of definitions. Consider the following simple example: a theory that contains a definition of a ! "$# predicate. %'&)(